feat: GPUBuffer.readSync() - synchronous small-buffer readback - #447
feat: GPUBuffer.readSync() - synchronous small-buffer readback#447mrousavy wants to merge 6 commits into
Conversation
Adds a non-spec extension for the GPU-compute readback pattern that mapAsync cannot serve: render/worklet loops that must consume a compute result in the SAME frame (hand/pose landmarks, histogram ranges, GPU picking ids, counters). Awaiting mapAsync from such a loop forces at least one frame of staleness; readSync blocks the calling thread until previously submitted work touching the buffer completes and returns an owned copy of the bytes. Implementation: MapAsync with CallbackMode::WaitAnyOnly + a 2s Instance::WaitAny - the instance already enables TimedWaitAny. External instances without the feature (e.g. Skia-provided) fail the wait and throw instead of hanging. Capped at 1 MiB: this is a primitive for tiny results, not bulk transfers; the async path remains the right tool there. Requires MAP_READ usage, so the usage rules push callers to the correct copy-to-staging pattern by construction. Typed via the existing non-spec declare-global block, with tests covering the same-tick compute readback, offset/size, repeated reads, and the size cap.
|
Interesting idea, is the 1 frame cadence a big deal? If I understand the use case is synchronizing the landmark info with the depth info? I feel like the readback is skippable or at least the synchronization is |
There was a problem hiding this comment.
I like it.
- Can we find a better name than OwnedArrayBuffer?
- Also isn't
readSyncmisleading if we are doing a full copy of the buffer? I guessreadSyncis an ok name, we are just reading, not mapping. - Shouldn't the timeout be an input parameter?
- Can we document it in
apps/docs. There are many native WebGPU APIs which we expose in this module already.
@reczkok made an interesting comment, I'm not sure what it means exactly?
It is, the stream feels much more sluggish if you have 1 frame delay for gesture driven recognition. Needs to be sync unfortunately, but I am wondering if we can pack state into the GPU |
|
but doesn't it make it slower if you block the thread to do a buffer copy? |
:)
Suggestions? It creates memory and will free it - so per definition it owns it. But Nitro
Suggestions? I think readSync is fine, we can also do copySync?
Good point, will add it!
Will do!
What do you mean by that? An |
Slower than what? What is the alternative? |
|
slower than being a frame delayed? |
|
How come? This runs in the same frame, and "frame delayed" runs in the next frame? |
|
I guess it's an architectural choice - if you want to not do any extra CPU syncs you can run it on the next Frame, but this will cause visual delay for stuff like hand tracking, even if it's very subtle. My approach is fully sync in the same Frame, so it looks much more snappy but causes an extra CPU sync. My machine has the budget for that so imo that is the better approach for my demo here https://x.com/mrousavy/status/2090591615639269829?s=20 |
|
ok sounds good. Let's name it |
3 participants
Motivation
There is a GPU-compute readback pattern that
mapAsynccannot serve: a render loop (or a react-native-worklets frame callback) that must consume a small compute result in the same frame it was produced. Examples: hand/pose landmarks driving what gets rendered this frame, histogram/range reductions feeding a later pass's uniforms, GPU picking ids, atomic counters. AwaitingmapAsyncfrom such a loop forces at least one frame of staleness, because the promise cannot resolve until the loop yields.readSync(offset?, size?)blocks the calling thread until all previously submitted work touching the buffer completes, then returns an owned copy of the bytes:Implementation
MapAsyncwithCallbackMode::WaitAnyOnly+ a boundedInstance::WaitAny- the library already creates its instance with theTimedWaitAnyfeature (timedWaitAnyMaxCount: 64), so no instance changes are needed. External instances without the feature (e.g. the Skia-provided instance path) fail the wait and throw instead of hanging; a hung GPU trips the 2s timeout rather than deadlocking JS.Deliberate limits that keep this a scalpel rather than a footgun:
mapAsyncremains the right tool for bulk transfersMAP_READusage, whose only valid companion isCOPY_DST- so the API pushes callers into the correct copy-to-staging pattern by constructionTyped in the existing non-spec
declare globalblock alongside the other RN extensions.Tests
ReadSync.spec.ts: same-tick compute-pipeline readback, offset/size sub-reads, repeated reads on one buffer, and the size-cap error.Real-world validation
Running in a VisionCamera + TypeGPU app that executes a monocular depth model (~260 compute dispatches) per camera frame inside a worklet: a 16-byte probe result (hand depth sampled from the disparity buffer) is read back with
readSyncand drives the same frame's lighting uniforms, at 30fps sustained on an M1 Max. Replacing the previousmapAsync-on-an-interval design removed one frame of control latency and all of the cross-runtime plumbing it required.🤖 Generated with Claude Code